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Abstract 

We study a special class of diamond channels which was introduced by Schein in 
2001. In this special class, each diamond channel consists of a transmitter, a noisy 
relay, a noiseless relay and a receiver. We prove the capacity of this class of diamond 
channels by providing an achievable scheme and a converse. The capacity we show 
is strictly smaller than the cut-set bound. Our result also shows the optimality of a 
combination of decode-and-forward (DAF) and compress-and-forward (CAF) at the 
noisy relay node. This is the first example where a combination of DAF and CAF is 
shown to be capacity achieving. Finally, we note that there exists a duality between 
this diamond channel coding problem and the Kaspi-Berger source coding problem. 
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1 Problem Statement and the Result 



The diamond channel was first introduced by Schein in 2001 [1]. The diamond channel 
consists of one transmitter, two relays and a receiver, where the transmitter and the two 
relays form a broadcast channel as the first stage and the two relays and the receiver form 
a multiple access channel as the second stage. The capacity of the diamond channel in its 
most general form is open. Schein explored several special cases of the diamond channel, one 
of which [1, Section 3.5] is specified as follows (see Figured]). The multiple access channel 
consists of two orthogonal links with rate constraints Ri and R 2 , respectively. The broadcast 
channel contains a noisy branch and a noiseless branch, i.e., with input X and two outputs 
X and Y . We refer to the relay node receiving Y as the noisy relay and the relay node 
receiving X as the noiseless relay. Schein provided two achievable schemes for this class of 
diamond channels. In this paper, we will prove the capacity of this special class of diamond 
channels. 

The formal definition of the problem is as follows. Consider a channel with input alpha- 
bet X and output alphabet y, which is characterized by the transition probability p(y\x). 
Assume an n-length block code consisting of (/, g,h,tp) where 

f:{l,2,...,M}»X n (1) 

g-.y n ^{i,2,...,\g\} (2) 

h:{l,2,...,M}^{l,2,...,\h\} (3) 
^:{l,2,...,M}x{l,2,...,|/i|}^{l,2,...,M} (4) 

Here / denotes the encoding function at the transmitter, g and h denote the processing 
functions at the noisy and noiseless relays, respectively, and ip denotes the decoding function 
at the receiver. 

The encoder sends x n = f(m) into the channel, where m £ {1, 2, ... , M}. The decoder 
reconstructs rh = tp(g(Y n ), h(m)). The average probability of error is defined as 

1 M 

P e = — Pr(rh 7^ m\m is sent) (5) 
m=l 

The rate triple (R, Ri, R2) is achievable if for every 0<e<l, ?7>0 and every sufficiently 
large n, there exists an n-length block code (/, g, h, <p), such that P e < e and 

- In M>R- 7] (6) 
n 

-\n\g\<R 1 + V (7) 

n 

-ln\h\<R 2 + V (8) 
n 
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Figure 1: The diamond channel. 

The following theorem characterizes the capacity of the class of diamond channels con- 
sidered in this paper. 

Theorem 1 The rate triple (R, Ri, R2) is achievable in the above channel if and only if the 
following conditions are satisfied 



R < I(U; Y) + H(X\U) 
Ri > I(Z;Y\U,X) 
R 2 > H{X\Z, U) 
R x + R 2 >R + I(Y\Z\X, U) 



(9) 
(10) 

(11) 
(12) 



for some joint distribution 



p(u, z, x, y) = p(u, x)p(y\x)p(z\u, y) 



(13) 



with cardinalities of alphabets satisfying 



\U\ < \X\ +4 

\z\ < \u\\y\ + ?>< 1^113^1 + 41^1 + 3 



(14) 
(15) 



2 The Achievability 

Assume a given joint distribution 

p(u, z, x, y) = p(u, x)p(y\x)p(z\u, y) (16) 

and consider that the information theoretic quantities on the right hand sides of ([9]), ( fTOlh 
(jlip and (|12p are evaluated with this fixed joint probability distribution. 

Consider a message W with rate R. If R < H(X\Z,U), reliable transmission can be 
achieved by letting g{Y n ) = <p (constant) and h(W) = W, i.e., by sending the message 
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R + I(Z; Y\U, X) - H(X\U, Z) 



I(U;Y)+I(Y;Z\U) 



I(Z;Y\U,X) 




b 



b' 



R — J(U;Y) — I(X;Z\U) H(X\U,Z) R R + I(Z; Y\U, X) 
Figure 2: Rate region of (R u R 2 ) when H(X\U, Z) < R < I(U; Y) + I(X; Z\U). 

through the noiseless relay. Thus, we will only consider the case where 



We will show that the message can be reliably transmitted with a pair of functions (g, h) 
such that (- In \g\, - In \h\) lies in the inverse pentagon^] with corners a and b in Figure [2j 
However, we instead prove reliable transmission with (- In |^|, - In \h\) lying in the inverse 
pentagon with corners a' and b' , which contains the inverse pentagon with corners a and 
b and thus imposes a stronger condition to prove. It is straightforward to have reliable 
transmission with the rate pair at point b' by letting g(Y n ) = <fi (constant) and h(W) = W. 
Thus, it remains to prove that reliable transmission is possible with the rate pair at point 
a', i.e., 



Let us assume that the message W is decomposed as W = (W a , W&, W c ). For a positive 

"inverse pentagon" with corner points a and b, we mean the region in the (Ri, R2) space that is to 
the "north-east" of line segment [a, b}. More specifically, this is the region described by inequalities in (I10[) . 



H(X\Z, U)<R< I{U; Y) + H(X\U) 



(17) 



R! = I(U) Y) + I(Y;Z\U) 

R 2 = R-I(U; Y)-I(X;Z\U) 



(18) 
(19) 



(fTTjl and |T2]). 
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number e, let us define 

M a = \W a \ = exp(n(I(U; Y) - 3e)) (20) 
M 

M b ±\W b \ = jjjj = exp(lnM - n(I(U; Y) + I(X; Z\U) + 6e)) (21) 

M c = \W C \ = exp(n(I(X; Z\U) - 3e)) (22) 



Random codebook generation: We use a superpostion code structure. The size of the 
inner code is M a . For each inner codeword, we independently generate M b outer codes. The 
size of each outer code is M c . 

• Independently generate M a sequences, u n (l), u n (2), . . . , u n (M a ), according to Y\a=i P( u i) 
where p(ui) = p(u), for % — 1, 2, . . . , n. 

• For u n (j), j = 1,2,..., M a , independently generate M b codebooks, C(j, l),C(j, 2), . . . , 
C(j,M b ). 

• In the codebook C(j, k), j = 1, 2, . . . , M a , k = 1, 2, . . . , M b , independently generate M c 
codewords x n (j, k, 1), x n (j, k, 2), . . . , x n (j, k, M c ) according to niLi^K 2 '*!^ = u iU))i 
where p(xi\U = Ui(j)) = p(x\u), for % — 1, 2, . . . , n, j = 1, 2, . . . , M a , k = 1, 2, . . . , M b . 

There will be no overlapping codebooks with high probability when n is sufficiently large, 
because 

- In M b M c < H(X | U) (23) 
n 

Encoding at the transmitter: Let W = (W a , W b , W c ) be the message. We send codeword 
X n = f(W a , W b , W c ) = x n (W a , W b , W c ) into the channel. 

Processing at the noisy relay: First, after having received Y n , seek 

U n = u n (W a ) E K(l), u n (2), . . .,u n (M a )} (24) 

such that 

(U n ,Y n )eT^ Y] (25) 

where the definition of strong typical set can be found in [2, Section 1.2]. If there is not any 
such U n , then let U n be an arbitrary sequence in {u n (l), u n (2), . . . ,u n (M a )}. Secondly, con- 
struct a conditional rate distortion code according to YYi=i P( z ii Vil^i) with encoding function 
g'(Y n , U n ) and \g'\ = L = exp(ra(I(Y; Z\U) + r)). Finally send U n and Z n = g'(Y n , U n ) to 
the destination, i.e., 

g(Y n ) = (U n , Z n ) (26) 

where 

\g\ = M a x L < exp(n(/(f/; Y) + I(X; Z\U) +r- 3e)) (27) 
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Processing at the noiseless relay: Let h(f(W a ,W c ,Wb)) = Wb where 



\h\ =M b = exp(lnM - n(I(U; Y) + I(X; Z\U) + 6e)) (28) 

Decoding: Decoder collects (U n , Z n ) from the noisy relay and Wb from the noiseless relay. 
The decoder seeks a codeword x n (W a , Wb,i) from the codebook C(W a , Wb) such that 

(x n (W a ,W b ,i),Z n ) eT$ m (U") (29) 

Probability of error: The error occurs when (U,X) 7^ (U,X). The average probability of 
error can be decomposed into 

Pr(E) < Pr{E x U E 2 U E 3 ) = Pr(Ex) + Pr(E 2 n E{) + Pr(E 3 n £?f n £ 2 C ) (30) 

where 

^([T^y^)^^ (31) 

E2= |J «y")e^ y] (32) 

M"^C/ n ,u"e{M n (l),« n (2),...,u n (A/ £l )} 

£3= U (^.^)e^(^) ( 33 ) 

We note that 

Fr^) < Fr(EP ^ Tjft) + Pr((Y», Z n ) $ T [YZ]u] (U n )) + Pr(X" $ T^ YZU] (Y n , Z\ U n )) 

(34) 

where 

• U n is generated in an i.i.d. fashion with probability p(u). Thus, when n is sufficiently 
large, we have 

Pr(U n i 7ft) < e (35) 

• Z n is a conditional rate distortion code for Y n conditioned on U n . Thus, when n is 
sufficiently large, L = exp(nI{Y; Z\U) + r), and U n G 7ft, we have 

Pr((Y n ,Z")tT$ zm (U n ))<e (36) 

• X n can be viewed as being generated according to an i.i.d. conditional probability 
p(x\u, y) with respect to (U n , Y n ). Thus, when n is sufficiently large and (Y n , Z n , U n ) G 



1 [YZU] ' 



Pr(X n <£T$ lYZU] (Y n ,Z n , U n ))< e (37) 
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From the above calculation, we have 

Pr(E x ) = Pr((U n , X n , Y n , Z n ) $ T$ xz] ) < 3e 
For the second error event, we note that M a = exp(n(I(U ; Y) — 3e) and 

Pr(E 2 n E{) = Prl \J («», Y n ) G T$ Y] \ (Y n ) G 7ft 

\ u n ^U n ,u n £{u n (l),u n (2),...,u n (M a )} 

M a 

<Y,Pr((u n (i),Y n ) ElfalY" €7fr) 

<M a Pr(u n (t)eT^ Y] (Y n )) 

< M a exp(-nH(U) + ne) exp(nH(U\Y) + ne) 
= exp(— ne) 

< e 

for sufficiently large n. We note that M c = exp(n(I(X; Z\U) — 3e), then 



Pr(E 3 n E?) = Pr |J (a* Z») G 7^ |[7] (t/")|(Z", [/) G 2g 



-n 

'-[ZU] 

,x n ^X n ,x n £C(W a ,W b ) 



M c 



< "£Pr((x(M a ,M b ,i),Z n ) G T { n xzlu] (U n )\(Z n ,U n ) G T { n zu] ] 



i=l 



< M c Pr(x(M a ,M b ,i) G T [xlzu] (Y n )) 

< M c exp(-nH(X\U) + ne) exp(nH(X\Z, U) + ne) 
= exp(-ne) 

< e 

for sufficiently large n. Thus, the average probability error is upper bounded as 

Pr(E) <3e + e + e = 5e 
which goes to zero when n goes to infinity 

3 The Converse 

Define Z< = g and U t = (Y*' 1 , We note that 

p(ui, Xi, Hi, Zi) = p(v,i, Xi)p{yi\Xi)p{zi\y h ui) 
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We have 

lnM = H(X n ) 

= Y,H(X l \X? +1 ) 

i=i 

n 

<Y,I{Y l ~ l ^) + H{X t \X? +1 ) 
i=i 

n 

= J2 HY*- 1 , X? +1 ; K t ) - J(Xr +1 ; F^" 1 ) + H{X t \Y^\ X? +1 ) + /(F^ 1 ; X,|Xf +1 ) 

i=l 
. . n 

i=l 
n 

= ^2l(U i ',Y^+H(X i \U i ) (43) 



i=l 

where 



1. Because of the following equality [3, Lemma 7] 

n n 

Y^nx^nY*- 1 ) = ^/(r^xix^) (44) 



i=l i=l 



s 



We have 



\n\g\>H(g) 

> H(g\h) 

>H(g\h)-H(g\h,Y n ) 
= I(g;Y n \h) 

n 

= J2Hg;Y*\h,Y 1 - 1 ) 

i=l 
n 

= Hg, ^r+i! Yi\h, Y'- 1 ) - Yi\g, h, 

i=l 
n 

S i(g, Yi\h, Y l ~ l ) - /(r- 1 ; X^, /i, 

> ]T /(<?, x? +l] Yi\h, r- 1 ) - # (x^, h, x% 



i=l 

n 

= -H(X n \g, h) + J2 ^9, JfS-il m 

i=l 

E] n 

i=l 
n 

>£%y i |A,y < - 1 ) ^ 1 )-6 

i=l 

E] " 

^/(^yi^y-Sx^jQ-e 
t=i 

. . n 

i^/^yiy-Sx^x^-e 

i=l 
n 

= ^2l(Z i -Y i \U i ,X i )-e (45) 



i=l 

where 

1. Because of the following equality [3, Lemma 7] 



£ y^, /i, y 1 ) = £ /(y- 1 ; X^, ft, Xf +1 ) (46) 



i=l i=l 



2. Due to Fano's inequality. 

3. g is a deterministic function of Y n . Due to the memoryless property, we have 

H{g\Y i ,h,Y i -\X». 1 ,X i ) = H{g\Y i ,h,Y i -\X& 1 ) (47) 



4. g is a deterministic function of Y n and h is a deterministic function of X n . Due to the 
memory less property, we have 



H(g\h,Y*-\X? +1 ,X i )=H(g\Y'- 1 ,XZ. 1 ,X i ) (48) 

n 

i+l> 



^I^F 4 - 1 ,^,^,^) =^|F i - 1 ,Xr +1 ,X i ,F J ) (49) 



We have 



ln|/i| >77(%) 

* n l<?) 

= 77pT%)-77pr%,/ i ) 

m 

>H(X n \g)-ne 

n 

= Y,H(X i \X? +1 ,g)-e 

i=l 
n 

>J2H(X i \Y i -\X? +1 ,g)-e 

i=i 

n 

= TT t H{X i \U i ,Z i )-e (50) 



i=l 



where 

1. Due to Fano's inequality. 
We have 



\n\g\ + \n\h\>H(g,h) 

>I(g,h;X n ,Y n ) 

>I(X n ;g,h) + I(Y n ;g,h\X n ) 

= H(X n ) - H(X n \g, h) + I(Y n ; g, h\X n ) 

m 

>hxM -ne + I (Y n ; g , h\X n ) 
ilnM-ne + I{Y n -g\X n ) 

n 

= InM + ~t + m,9\X n , Y*- 1 ) 
i=i 

n 

I i n M + £ -e + /(y i; r- 1 , Xf +1 ) 

1=1 

n 

= In M + ^ -e + 7(y i; £/,) (51) 



8=1 
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1. Due to Fano's inequality. 

2. h is a deterministic function of X n 

3. g is a deterministic function of Y n . Due to the memoryless property, we have 

H{g\X t ^-\X^X 1 - 1 ) = H{g\X u Y*-\X? +1 ) (52) 
H(g\Y i} X iy Y*- 1 , X*" 1 ) = fT(y|y<, X J7 (53) 

We note that - In M > i? — 77, ^ In |g| < -Ri + r/ and ^ In \h\ < R2 + r), for an arbitrary 
7] > 0. Assume e — ► 0, then from (J4"3j) , fH5|) . fl50l) and (j5Tj) . we have 

1 " 

i?< _ V /([/,; y) + if (X,|^) (54) 
n z — ' 

i=l 

1 - 

Ri > -J^riZtYilU^Xi) (55) 
i=i 

^^^W,,^) (56) 

i=l 

1 " 

i2 1 + i? 2 > J R+-V/(y i ;Zi|X i ,f/ i ) (57) 



n . 



Define a time-sharing random variable Q, which is uniformly distributed on {1,2, ... ,n}. 
Also define a set of random variables (X, Y,U , Z) such that 

Pr(X = x,Y = y,U = u, Z — z\Q = 1) = p(Xi = x, Yi = y,Uj = u,Z { = z), i = 1, 2, . . . , n 

(58) 

Define U = (U, Q) and Z = (Z, Q), then 

1 - 

R < -Y I{Ui\Yi) + H (Xi I Ui) 

i=l 

= I(U;Y\Q)+H(X\U,Q) 
< I{U,Q;Y)+H{X\U,Q) 

= I{U;Y) + H{X\U) (59) 



1 n 

Ri >-'Yl(Z i ;Y i \U i ,X i 



n . 



I(Z;Y\U,Q,X) 

I(Z;Y\U,X) (60) 
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R 2 >-^H(X i \U i ,Z i ) 
n ^— ' 

i=l 

= H{X\U,Z,Q) 

= H(X\U,Z) (61) 



1 

i?x + i? 2 > i? + - V /(y i; t/,) 

n 

= R + I(Z;Y\U,X,Q) 

= R + I(Z;Y\U,X) (62) 

where fl5^]h (1501). (IBTT) and (IS2l are the same as (151), (flUl). (1TT1) and (1T21, concluding the proof. 

Finally, we note that the bounds on the cardinalities of the alphabets in (fT4l) and (|T5l) 
can be proven in a way similar to [4, Appendix D]. 

4 Remarks 

We have several remarks regarding this result as follows: 

1. The capacity is strictly smaller than the cut-set bound [5], because first 

R<R 1 + R 2 -I(Y;Z\U,X) (63) 

An operational interpretation is that when the noisy relay cannot fully decode the 
message, or in other words, when the noisy relay cannot remove the noise completely, 
the data going through the link from the noisy relay to the receiver contains noise. 
Thus, the useful information flowing through the multiple access cut will be strictly 
less than Ri + R 2 . Secondly, we note that 

R < I(U;Y) + H(X\U) < H(X) (64) 

An operational interpretation is that when the noisy relay decodes the message with 
a positive rate, the rate of information flowing through the broadcast cut becomes 
strictly less than H(X). 

Consider the following example. Let X and Y be binary and 

Y = X © W (65) 

where the sum is a modulo-2 sum and W has a Bernoulli distribution with entropy 
0.5 bits. We assume R\ — R 2 — 0.5 bits. The cut-set bound in this example is 1 bit, 
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which is not achievable. Because if R is equal to 1 bit, we have, 



R = I(U; Y) + H{X\U) = H(X) = 1 



(66) 



then, U has to be independent of X and Y. Also, we have 



R = Rx + R 2 - I(Y; Z\U, X) = R x + R 2 = 1 



(67) 



then, Z has to be independent of X and Y if U is independent of X and Y. However, 
if U and Z are independent of X and Y, we arrive at the following contradiction, 



which means that the cut-set bound is not achievable in this example. We note that, 
even in this binary example where \X\ = \y\ = 2, the cardinalities of the auxiliary 
random variables U and Z are \U\ < 6 and \Z\ < 15. These large cardinality bounds 
make it practically impossible to evaluate the capacity of this diamond channel. How- 
ever, we note that, even though we were not able to compute the exact value of the 
capacity in this example, we were able to conclude that the capacity is strictly less 
than the cut-set bound, which is 1 bit. 

We know that the capacity of a diamond channel with four orthogonal links is equal 
to the cut-set bound in this channel. Our result shows that introducing the broadcast 
node will reduce the capacity of this all-orthogonal diamond channel. Networks with 
broadcast nodes have been studied recently from different perspectives, e.g., informa- 
tion theory and network coding [6-8]. We note that our diamond channel model is a 
simple example of a general network with a broadcast node. Thus, we conclude that 
the cut-set bound in general is not tight in networks with broadcast nodes. 

2. The processing at the noisy relay includes two operations: decode the inner code U n and 
compress the channel output Y n to Z n conditioned on U n . This processing is essentially 
the same as Theorem 7 in [9], i.e., combination of DAF and CAF. DAF [9, Theorem 1] 
has been shown to be optimal in the degraded relay channel [9]. Partial DAF, a special 
case of [9, Theorem 7] without compression, has been shown to be optimal in semi- 
deterministic relay channel [10] and the relay channel with orthogonal transmitter-relay 
link [11]. Recently, CAF [9, Theorem 6] has been shown to be optimal in two special 
relay channels [12,13]. To our knowledge, we are the first to show the optimality of 
the combination of DAF and CAF in some specific channel, even though the channel 
we consider is not a three-node relay channel in the strict sense, i.e., as in [9]. 



0.5 = R 2 > H(X\Z, U) = H(X) = 1 



(68) 
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Decoder 



(Relay 2) 



Figure 3: Kaspi-Berger rate distortion problem. 

3. If we assume R = H(X) — Rq, then Theorem [1] can be rewritten as follows 

R < I(U;Y) +H(X\U) < — > R >I(U;X\Y) (69) 

R x >I{Z-Y\U,X) < — > R 1 >I(Z;Y\U : X) (70) 

R 2 > H(X\Z, U) < — R 2 > I(X; X\Z, U) (71) 

R x + R 2 > R + I{Y; Z\X, U) < — > R + R 1 + R 2 >I(X,Y-U,X,Z) (72) 

for some joint distribution 

v{% z, x, y) = p(u, x)p(y\x)p(z\u, y) (73) 

We note that the right hand sides of ([69]), {70}, (EE} and (E2J) in addition to the 
distribution constraint in (1731 are the same as the rate region of the rate-distortion 
problem studied by Kaspi and Berger as shown in Figure[3] [4, Theorem 2.1, Case C]. 

This duality between our diamond channel coding problem and the Kaspi-Berger source 
coding problem is similar to the duality between the single-user channel coding problem 
and the Slepian-Wolf source coding problem [2, Section 3.1] by viewing the codebook 
information in the channel coding problem as the information sent to all the terminals 
in the source coding problem, e.g., the information with rate Rq in Figure [3j Thus, the 
achievability of our diamond channel coding problem can be obtained from the achiev- 
ability of Kaspi-Berger source coding problem, in the same way that the achievability 
of the multiple access channel coding problem can be obtained from the achievability 
of fork network coding problem [2, Section 3.2]. 



11 



References 



[1] B. E. Schein. Distributed Coordination in Network Information Theory. PhD thesis, 
Massachusetts Institute of Technology, 2001. 

[2] I. Csiszar and J. Korner. Information Theory: Coding Theorems for Discrete Memory- 
less Systems. Academic Press, 1981. 

[3] I. Csiszar and J. Korner. Broadcast channels with confidential messages. IEEE Trans. 
Inform. Theory, 24(3):339-348, 1978. 

[4] A. H. Kaspi and T. Berger. Rate-distortion for correlated sources with partially sepa- 
rated encoders. IEEE Trans. Inform. Theory, 28(6):828-840, 1982. 

[5] T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley and Sons, 
1991. 

[6] A. F. Dana, R. Gowaikar, R. Palanki, B. Hassibi, and M. Effros. Capacity of wireless 
erasure networks. IEEE Trans. Inform. Theory, 52 (3): 789-804, 2006. 

[7] N. Ratnakar and G. Kramer. The multicast capacity of deterministic relay network 
with no interference. IEEE Trans. Inform. Theory, 52(6):2425-2432, 2006. 

[8] G. Kramer, S. M. S. Tabatabaei Yazdi, and S. A. Savari. Network coding on line 
networks with broadcast. In Proc. Conf. Inf. Sciences and Systems (CISS), Princeton, 
NJ, Mar. 2008. 

[9] T. M. Cover and A. El Gamal. Capacity theorems for the relay channel. IEEE Trans. 
Inform. Theory, 25:572-584, Sep. 1979. 

[10] A. El Gamal and M. Aref. The capacity of the semideterministic relay channel. IEEE 
Trans. Inform. Theory, 28(3) :536, 1982. 

[11] A. El Gamal and S. Zahedi. Capacity of a class of relay channels with orthogonal 
components. IEEE Trans. Inform. Theory, 51(5):1815-1817, 2005. 

[12] Y. H. Kim. Capacity of a class of deterministic relay channels. IEEE Trans. Inform. 
Theory, 53(3):1328-1329, 2008. 

[13] M. Aleksic, P. Razaghi, and W. Yu. Capacity of a class of modulo-sum re- 
lay channels. Submitted to IEEE Transactions on Information Theory, 2007, 
http://arxiv.org/pdf/0704.3591. 



15 



